Skip to content

DEV-1262: while streaming, complete the command in flight only on its own in-stream reply - #330

Merged
JongChern merged 1 commit into
masterfrom
DEV-1262_instream_response_queue
Oct 21, 2026
Merged

JongChern merged 1 commit into
masterfrom
DEV-1262_instream_response_queue

Conversation

@JongChern

@JongChern JongChern commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Fixes DEV-1262. This is the driver half of DEV-1149, whose Android half is ShimmerResearch/ShimmerAndroidAPI#122.

Defect

While streaming, the branch of ShimmerBluetooth.processPacket() that handles an in-stream response (DATA … FF 8A <opcode> …) did four things, whatever the response was:

  • cancelled the ACK timer;
  • cleared mWaitForAck and mWaitForResponse;
  • called removeInstruction(0);
  • unlocked the instruction queue.

The firmware also sends a status response unprompted: on a dock or undock, the user button, a trial-duration expiry or a low-battery stop. When one of these arrived:

  • while a SET command was waiting for its ACK, that command was removed, and its ACK was then matched to the next instruction. For startSDLogging() (SET_RWC, then START_LOGGING_ONLY), that means processAckFromSetCommand(SET_RWC) runs against START_LOGGING_ONLY's one-byte instruction;
  • while a GET command was waiting for its reply, that command was completed early;
  • with nothing in flight, a queued instruction not yet sent was removed.

The protocol matches a reply to its command by the response opcode, never by the ACK: log-and-stream-common docs/SHIMMER3_BT_COMMUNICATION_PROTOCOL.md, section 5.4, rule 2.

Fix

  • The streaming branch reads the response opcode first. It completes the command in flight only if a command is waiting and the opcode is that command's expected response in mBtGetCommandMap (isReplyToCommandInFlight()).
    • For a reply, the timer is cancelled and the flags cleared before parsing, as before.
    • Any other response is still processed, but leaves the queue, the wait flags and the ACK timer alone.
  • processInstreamResponse(boolean) keeps its signature. It reads the opcode and hands it to a new processInstreamResponse(byte, boolean), which holds the unchanged parsing. Subclass overrides and the other caller (the not-streaming path) keep working. ShimmerAndroidAPI's Shimmer overrides the boolean version.
  • Size: ignoring whitespace, the driver change is +36/−6. The raw diff is larger because the parsing body moved out one nesting level.

Tests

New API_00030_StreamingInstreamResponseQueueTest (9 tests) in ShimmerDriverPC:

  • It uses API_00027's in-memory harness: no hardware, and bytes fed to the parser one at a time.
  • Each scenario runs with the link CRC off, 1-byte and 2-byte.
Scenario Checks
status push while SET_RWC awaits its ACK (startSDLogging()) both instructions stay queued and the lock, flags and timer are unchanged; the later bare ACK completes SET_RWC and leaves START_LOGGING_ONLY queued
status push while GET_VBATT awaits its reply GET_VBATT keeps waiting; its own reply then completes it and the battery is parsed
status push with nothing in flight and GET_VBATT queued GET_VBATT stays queued

Each scenario also checks that the push itself is still processed: its docked bit takes effect.

Run Result
API_00030 on master 9 of 9 fail, at the queue-head removal
API_00030 with this fix 9 of 9 pass
API_00027 (DEV-1147), before and after 10 of 10 pass
ShimmerDriver gradlew test BUILD SUCCESSFUL, 0 failures
ShimmerDriverPC gradlew test BUILD SUCCESSFUL, 0 failures

Not run on hardware. The Android HIL (ShimmerAndroidAPI#128) can cover this over BLE once an Android build uses a driver with this fix.

The same rule in the other implementations

Implementation Verdict
This repo: comms/radioProtocol/LiteProtocol (in-stream branch of its packet parser) Same defect, not fixed here. It's reached through CommsProtocolRadio, which Consensys uses only for non-Shimmer3 devices (Sweatch, Shimmer4 SDK); Shimmer3 units go through ShimmerPCMSS → ShimmerBluetooth. Worth its own change if that path is still wanted
ShimmerAndroidAPI, Classic (Shimmer.java's own parser) same defect; fixed by ShimmerAndroidAPI#122
ShimmerAndroidAPI, BLE (Shimmer3BLEAndroid) uses this driver's parser, so it gets this fix through a driver release and pin
Shimmer-C-API (every copy) not applicable: commands are written and forgotten, with no queue or in-flight command
shimmer-web-sdk not applicable: no in-stream GETs while streaming, and non-data bytes are resynced away. A weaker form of rule 2 exists: on Shimmer3R BLE, any leading 0xFF resolves the awaited ACK, if a SET is sent while streaming
SwiftAPI not applicable: only STOP is in flight while streaming
pyshimmer already correct: a status response counts as a reply only if GetStatusCommand is in flight, with a test. A residual risk exists only with the ACK prefix left on
shimmer-logandstream not applicable: only STOP can be in flight while streaming

Release

Not urgent for Android 3.2.11_beta. It reaches each consumer as follows:

  • PC/Consensys and MATLAB: directly.
  • Android over BLE: through the next driver release, and an Android release built on it.

🤖 Generated with Claude Code

While streaming, the branch of processPacket() for an in-stream response
(FF 8A ...) cancelled the ACK timer, cleared the wait flags, removed the head
of the instruction queue and unlocked it, whatever the response was. The
firmware also sends a status response unprompted (dock, undock, the user
button, a low-battery stop). So such a push removed a SET command still
waiting for its ACK, or a queued instruction not yet sent, and the next ACK
was matched to the wrong instruction: for startSDLogging(),
processAckFromSetCommand(SET_RWC) against START_LOGGING_ONLY's one-byte
instruction.

The branch now reads the response opcode first and completes the command in
flight only if the opcode is that command's expected response
(mBtGetCommandMap), as the protocol says (log-and-stream-common
docs/SHIMMER3_BT_COMMUNICATION_PROTOCOL.md section 5.4, rule 2). Any other
response is still processed, but leaves the queue, the wait flags and the ACK
timer alone. processInstreamResponse(boolean) still reads the opcode itself
and hands it to the new processInstreamResponse(byte, boolean), which holds
the unchanged parsing, so overrides and other callers keep working.

API_00030 (9 tests, CRC off/1/2) covers a status push while a SET command
awaits its ACK, while a GET command awaits its reply, and with nothing in
flight. All 9 fail without the fix. ShimmerDriver and ShimmerDriverPC test
suites pass.

The Android half is ShimmerResearch/ShimmerAndroidAPI#122 (DEV-1149).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Copilot AI lite review requested due to automatic review settings October 21, 2026 05:36

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing. Review the Copilot code review billing settings.

@JongChern
JongChern merged commit 0e28afb into master Oct 21, 2026
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants